{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://pymt.readthedocs.io\"><img style=\"float: left\" src=\"../../media/powered-by-logo-header.png\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Why *pymt*?\n",
    "\n",
    "*pymt* provides a standard, easy-to-use interface to a wide range of models. *pymt* solves\n",
    "several problems often encountered when a user wants to find, run, and/or couple\n",
    "models to one another. Below I'll go through some of the problems a model user often\n",
    "encounters when trying to find and try out a new model. This is certainly not an exhaustive\n",
    "list.\n",
    "\n",
    "Can you think of other issues a user may encounter in trying to use and/or couple\n",
    "a model?\n",
    "\n",
    "Problems:\n",
    "* [Source code](#Getting-source-code)\n",
    "* [Compiling the source code](#Compiling)\n",
    "* [Documentation](#Documentation) (or lack thereof)\n",
    "* [Running a model](#Running-a-model)\n",
    "* [Debugging](#Debugging)\n",
    "* [Model coupling](#Model-coupling)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting source code\n",
    "\n",
    "Even if a user doesn't *need* source code to run a model (i.e. they been given\n",
    "a binary program they can simply run), it's still important for them to be\n",
    "able to have access to it: they may want to modify it, or have a closer look\n",
    "under-the-hood to see what's going on.\n",
    "\n",
    "#### Problem\n",
    "\n",
    "This is certainly less of a problem nowadays. However, it can still be an issue.\n",
    "A would-be model user has heard of some mysterious model and would like to try\n",
    "it out but can't find the source code. Instead, they are left trying to\n",
    "find the email (phone? fax?) for the *master-of-the-code* and then trying to convince that person\n",
    "to send a floppy disk of the source code to you. It's hard to believe, but this used to\n",
    "be a thing.\n",
    "\n",
    "#### Solution\n",
    "\n",
    "All of the models within *pymt* are open source. For the most part, the source code for\n",
    "*pymt* models are available in repositories on\n",
    "[GitHub](https://github.com) but some may be housed on other publicly\n",
    "accessable websites like [bitbucket](https://bitbucket.com), and\n",
    "[SourceForge](https://sourceforge.com). We don't enforce the use of *GitHub* but\n",
    "we do use it extensively for our code and it seems to be the most widely used\n",
    "version control system in our community. In any case, the source code for all of the\n",
    "models are freely available and not located behind a gate keeper. \n",
    "\n",
    "CSDMS maintains a database of model metadata on its website:\n",
    "* https://csdms.colorado.edu\n",
    "This is not the source code but descriptions of models that have been contributed\n",
    "by the community to CSDMS. Here you can query model by, for example, type, process,\n",
    "or name.\n",
    "\n",
    "The source code for many of the contributed models is on *GitHub* under the\n",
    "*csdms-contrib* organization.\n",
    "* https://github.com/csdms-contrib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compiling\n",
    "\n",
    "You have the FORTRAN source code, it looks great (maybe you've even modified it), but now you need to\n",
    "be to run it.\n",
    "\n",
    "#### Problem\n",
    "\n",
    "Depending on the model, this step may not be much of a problem. However, oftentimes this\n",
    "is the *biggest* problem a user encounters when trying to run a new model. At it worst,\n",
    "this step is extremely painful and is often enough of a hurdle that this is where the\n",
    "user stops.\n",
    "\n",
    "After getting the source code, there are still several issues to solve:\n",
    "* Do I have the necessary compilers installed on my target platform?\n",
    "* Was the code ever intended to be built on my target platform?\n",
    "* Do I have all of the necessary dependencies installed? If not, this step is\n",
    "  increased in complexity by the number of dependencies needed to install.\n",
    "\n",
    "\n",
    "#### Solution\n",
    "\n",
    "All of the models distributed in *pymt* come pre-compiled on a range of platforms. Although we\n",
    "distribute *pymt* through *Anaconda*, which is primarily a Python package manager, models\n",
    "written in C, C++, or FORTRAN are also available. Although we try to build all of the models\n",
    "for Linux, Mac, and Windows, all of the models aren't yet available on all of those platforms.\n",
    "We're trying though! Most all of them are available on Linux, and Mac.\n",
    "\n",
    "We use [conda-forge](https://github.com/conda-forge) to build and distribute models that\n",
    "can then be installed using *Anaconda*. One nice side effect of this is that we also provide\n",
    "a recipe describes how each piece of software is built so that you can do it yourself, if\n",
    "you need to.\n",
    "\n",
    "And, as always, we accept pull requests! If you've built a model on a platform that we\n",
    "haven't, we would love to hear from you."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Documentation\n",
    "\n",
    "Perhaps the number one thing that keeps a user from experimenting with a new model. Is a\n",
    "lack of documentation. A user has found some source code but there is no documentation.\n",
    "Unfortunately, we are seldom paid to write documentation. Instead, we are funded to write\n",
    "some code to solve a specific problem and that's it. Encountered with a new mode, if a\n",
    "user isn't told how to build, run, or modify a model, they'll most likely just move on.\n",
    "\n",
    "#### Problem\n",
    "\n",
    "A collection of source files without any documentation.\n",
    "\n",
    "#### Solution\n",
    "\n",
    "Both [*landlab*](https://landlab.readthedocs.io) and [*pymt*](https://pymt.readthedocs.io)\n",
    "are well documented. And, as such, a user using a model from either of these\n",
    "frameworks is able to tap into a either of these documentation bases for information\n",
    "about how to get and run a model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Running a model\n",
    "\n",
    "Getting, compiling, and installing a model isn't all that useful it the model can't be run.\n",
    "\n",
    "#### Problem\n",
    "\n",
    "Every model has it's own idosyncratic way of running. For example, model generally have\n",
    "model-specific input/output files, or command line arguments (or a GUI?). Or, even worse,\n",
    "there isn't an input file: instead, input parameters are changed in the source code and\n",
    "the code recompiled.\n",
    "\n",
    "\n",
    "#### Solution\n",
    "\n",
    "All of the models in *pymt* have a uniform interface (based on the [Basic Model Interface (BMI)](https://bmi.readthedocs.io)). This means that if you know how to use one *pymt* model, you know how\n",
    "to use all *pymt* models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Debugging\n",
    "\n",
    "It looks like there may be a problem with a model but you're not sure. Or, you are sure,\n",
    "but you don't know where exactly the problem lies.\n",
    "\n",
    "#### Problem\n",
    "\n",
    "Trying to track down bugs can be a difficult process: particularly in codes written in compiled\n",
    "languages like C or FORTRAN. Debugging oftentimes means inserting a bunch of print statements\n",
    "into the code, recompiling, examining the output, and repeating.\n",
    "\n",
    "#### Solution\n",
    "\n",
    "Because *pymt* models are written in Python, a user can interactively run a model. A model can\n",
    "be updated one time step at a time, it's state examined (perhaps using Python tools like numpy\n",
    "or plotted using *matplotlib*), or even changed, and then updated for another time step. This\n",
    "ability to debug a model by playing with it in Python has proven to be a valuable way to,\n",
    "not only get a feel for how a model works, but also to see if it's working properly or as\n",
    "expected."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model coupling\n",
    "\n",
    "A user would like to couple two models (or a model to a dataset). \n",
    "\n",
    "#### Problem\n",
    "\n",
    "There are potentially many.\n",
    "\n",
    "Some examples:\n",
    "* Models are written in different languages\n",
    "* Models don't provide a way for another model to access it\n",
    "\n",
    "Can you think of others? Perhaps some that you've encountered? \n",
    "\n",
    "#### Solution\n",
    "\n",
    "The *pymt* brings models from different languages (currently C, C++, FORTRAN, and Python)\n",
    "into a Python environment. Because of the BMI (*pymt* models all expose a BMI), we can\n",
    "mostly automate this process. Through *pymt* users are able to write Python scripts that\n",
    "run single models, or multiple models together.\n",
    "\n",
    "We'll show you how to do this today."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The *pymt* model library\n",
    "\n",
    "All of the models that are available through *pymt* are held in a Python module that you can import, `pymt.models`.\n",
    "\n",
    "To have a look at what models are currently available, we'll import the library\n",
    "and print the names of all of the models.\n",
    "\n",
    "For more information you can look at the [pymt documentation](https://pymt.readthedocs.io)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pymt.models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll now have a closer look at a model and see how a *pymt* model works. Rememeber, because\n",
    "*pymt* models all have the same interface, and so if you know how to use one, you'll know\n",
    "how to use all of them.\n",
    "\n",
    "Let's begin by picking a model from the above list. Pick one that sounds interesting to you."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Model = pymt.models.Plume # <- type the name of the model you would like to use"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Model` is now a class of the model that you've chosen. You could create multiple instances\n",
    "of a `Model` but until it's an instance, you can't do too much with it, so let's\n",
    "create an instance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Model()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now examine the model a little more. For instance, we can use the `help` function\n",
    "to get some information about the model. This will give us a brief summary of the model,\n",
    "the author, a version number, license, references, etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "help(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Scroll down a little in the help message and have a look at the *Parameters* section. These\n",
    "are input parameters to the model. That is, things that are set at the *beginning* of\n",
    "the model and cannot be changed thereafter. You can also get a view of them programmatically\n",
    "using the *parameters* attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for name, value in model.parameters:\n",
    "    print(f\"{name} [default = {value}]\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The lifecycle of a model\n",
    "\n",
    "Running a model in *pymt* involves four steps:\n",
    "* [setup](#setup): prepare input files\n",
    "  * *setup*\n",
    "* [initialize](#initialize): read input files\n",
    "  * *initialize*\n",
    "  * *input_var_names*\n",
    "  * *output_var_names*\n",
    "  * *var*\n",
    "  * *get_value*\n",
    "* [update](#update): advance one time step\n",
    "  * *update*\n",
    "  * *start_time*\n",
    "  * *time*\n",
    "  * *end_time*\n",
    "* [finalize](#finalize): shutdown\n",
    "  * *finalize*\n",
    "\n",
    "Below we'll briefly go through each of these steps."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setup\n",
    "\n",
    "Before a model can be run, it's input files must be prepared. If you haven't done this manually\n",
    "(which you are definitely free to do), you can use the model's *setup* method to help with\n",
    "this.\n",
    "\n",
    "By default, *setup* will create a temporary folder with files containing default values. However,\n",
    "for this example we'll specify a folder so that we can see what's going on. The following will create\n",
    "a new folder, *_my_model* (you can call it whatever you like), and, in that folder, will\n",
    "be model-specific input files. *setup* return a tuple that gives the name of the main configuration\n",
    "file and the full path name of the folder. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "config_file, config_folder = model.setup(\"_my_model\")\n",
    "print(f\"Input files are located here: {config_folder}\")\n",
    "print(f\"The main configuration file is: {config_file}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To double-check that something was actually done, you can use shell commands (hint: `ls`, and `cat`)\n",
    "to see what files were created and what their contents are. If you don't like the shell, you can\n",
    "always use the Jupyter tree-view.\n",
    "\n",
    "The set of files that you just created depend completely on the model you chose. All of these files\n",
    "a model-specific. However, note that we all used the exact same command to create them - regardless\n",
    "of our chosen model.\n",
    "\n",
    "Now, let's now change an input parameter. This is done through passing keywords to *setup*. The\n",
    "keywords that you can use are specific to a model and can be found through *help*. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.setup(\"_fast_river\", river_mouth_velocity=2.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You'll now see a new set up input files. If you look closely, you should be able to see your\n",
    "change.\n",
    "\n",
    "A couple of notes about the *setup* method.\n",
    "\n",
    "***Note***: It's not strictly required to run *setup* before *initialize* - it's just a\n",
    "conveient way to get a set of input files. One pattern that is sometimes used is to use\n",
    "*setup* to get a base set of input files and then edit some of the files by hand.\n",
    "\n",
    "***Note***: It's not strictly required that you run *initialize* at all - sometimes\n",
    "*setup* is the goal. For example, *setup* provides an easy way to programmatically\n",
    "create a large number of input files for, say, a Monte Carlo simulation.\n",
    "\n",
    "```python\n",
    ">>> from itertools import product\n",
    ">>> velocity_samples = [0.5, 1.0, 1.5, 2.0, 2.5]\n",
    ">>> width_samples = [100.0, 200.0, 300.0, 400.0, 500.0]\n",
    ">>> for n, (velocity, width) in enumerate(product(velocity_samples, width_samples)):\n",
    "...     model.setup(f\"_sim-{n}\", river_mouth_width=width, river_mouth_velocity=velocity)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from itertools import product\n",
    "\n",
    "velocity_samples = [0.5, 1.0, 1.5, 2.0, 2.5]\n",
    "width_samples = [100.0, 200.0, 300.0, 400.0, 500.0]\n",
    "\n",
    "for n, (velocity, width) in enumerate(product(velocity_samples, width_samples)):\n",
    "    model.setup(f\"_sim-{n}\", river_mouth_width=width, river_mouth_velocity=velocity)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Initialize\n",
    "\n",
    "Now that we have a set of input files, we're ready to get the model ready for time stepping. This is\n",
    "done through the *initialize* method. The model is not in a state that we can query it, until\n",
    "*initialize* has been run. This is important to understand.\n",
    "\n",
    "To better understand this, consider the common pattern for a model to read from one of its input\n",
    "files the size or resolution of its solution grid. Thus, in such a situation, we cannot ask\n",
    "about the model's grid until its read input files, which is done in *initialize*.\n",
    "\n",
    "To run *initialize*, we must pass the name of a configuration file and a folder - both of\n",
    "which we got from *setup*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.initialize(config_file, config_folder)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can ask some questions about the model:\n",
    "* what variables do you provide as output?\n",
    "* what variables to you use as input?\n",
    "* what is the grid like on which these variables sit (if there even is a grid)?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Input and output variables\n",
    "\n",
    "Input and output variables are different from the parameters we talked about above (the ones described\n",
    "in the *help* message, or the *model.parameters* attribute). Input and output variables are\n",
    "able to ***dynamically change with time***.\n",
    "\n",
    "To get a list of the available input and output variables, you can use the *input_var_names* and\n",
    "*output_var_names* attributes. Depending on the model you chose, you may have not have any\n",
    "input variables. This means your model is configured once at the start but then can't be changed.\n",
    "It could be part of a 1-way coupling but not a 1-way coupling with feedback. A *dataset* would\n",
    "also be an example of a model without input variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Input variables:\")\n",
    "for name in model.input_var_names:\n",
    "    print(f\"- {name}\")\n",
    "\n",
    "print(\"Output variables:\")\n",
    "for name in model.output_var_names:\n",
    "    print(f\"- {name}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These are just the names of the variables. We can get additional information as well. This can\n",
    "be obtained several ways but the preferred method is using the `var` attribute. `var` is\n",
    "a dictionary of variables names mapped to variable descriptions.\n",
    "\n",
    "Pick a variable from the above list to find out more about it. We see attributes of the variable\n",
    "such as its data type and units. This also gives us information about the grid that the\n",
    "variable is defined on (i.e. the *grid* and *location* attributes). We'll get to grids later on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "variable = model.var[\"sea_bottom_sediment__deposition_rate\"] # <- replace this string with a variable for your model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "variable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can get it values either with the `data` attribute or with the `get_values` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "variable.data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.get_value(\"sea_bottom_sediment__deposition_rate\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Run\n",
    "\n",
    "Our model is now initialized and ready to be advanced through time. The *update* method advances\n",
    "the model's state by a single time step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.update()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's it. There not too much to it. You can see that it's done something either by seeing if\n",
    "an output variable has changed or using the *time* attribute to see the current model time -\n",
    "if there is one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Start time: {0} {1}\".format(model.start_time, model.time_units))\n",
    "print(\"Current time: {0} {1}\".format(model.time, model.time_units))\n",
    "print(\"End time: {0} {1}\".format(model.end_time, model.time_units))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Finalize\n",
    "\n",
    "There's not much to this method, and often we don't even use it. This is where a model will free memory\n",
    "or close files. If you're model uses lots of memory and you notice you're running out, it may\n",
    "help to call this method. Calling *finalize* will put your model in a state where it is no\n",
    "longer usable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.finalize()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
